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The Data Quality Campaign is a national, collaborative effort to encourage and support state 
policymakers to improve the collection, availability and use of high-quality education data 
and to implement state longitudinal data systems to improve student achievement. The 
campaign aims to provide tools and resources that will assist state development of quality 
longitudinal data systems, while also providing a national forum for reducing duplication of 
effort and promoting greater coordination and consensus among the organizations focusing 
on improving data quality, access and use. 


To these ends, four site visits were conducted in the spring of 2006 to state education 
agencies (SEAs) to gather information on their experiences in developing statewide 
longitudinal data systems: Florida, Utah, Virginia, and Wisconsin. 


This publication was produced by the Data Quality Campaign/National Center for 
Educational Accountability for submission under contract with the Bill & Melinda Gates 
Foundation. The report was written by Cherry Kugle and Nancy Smith and was reviewed and 
approved by the Florida Department of Education. 


© Copyright 2006. Data Quality Campaign. All rights reserved. 
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Florida: Evolving and I mproving 

The Florida Department of Education (FDOE) oversees 67 county- wide public school 
districts, encompassing almost 4,000 schools, in which over 2.6 million students were 
enrolled in the fall of the 2005-06 school year. Florida has a long history of collecting a 
multitude of data on its public education system and was one of the country's pioneers in 
collecting student- level data. 

History of Development 

Legislation 

Early efforts to evaluate and hold educational programs accountable in Florida began with 
legislation passed in 1968 instructing the Department of Education to improve educational 
effectiveness. The Florida Statewide Assessment Program was created as a result of the 
1971 Educational Accountability Act. An important element in the state's accountability 
effort, the program was designed to assess students' academic strengths and weaknesses, 
particularly in the basic skills. Since 1984, accountability for career and technical education, 
especially at the postsecondary level, has also been a focus in Florida. Accountability 
systems for community colleges and the state university system have been required by 
state statute since 1991. The state legislature in Florida has historically been supportive of 
implementing and enhancing statewide student longitudinal data systems for informing and 
improving public education. Currently, in every legislative budget a portion of the funding 
allocated to school districts must be used for data and information services. 

Data Collection Processes 

In 1986-87, Florida piloted collecting individual student-level data through the Florida 
Information Resource Network (FIRN). FIRN was a mechanism provided for districts to 
transmit data from the systems they were using locally. The data collected via FIRN were 
compared to the aggregate data collected in summary reports. Confidence and 
improvements in the new data collection system were built over several years before the 
summary data collection was eliminated. In 1990, the FDOE began to use the data 
collected through FIRN for reporting on the P-12 education system. 

In 1988 the Florida Education and Training Placement Information Program (FETPIP) was 
implemented. The FETPIP is a data collection system that obtains follow-up information on 
students after they exit the P-12 system and includes employment, postsecondary 
education, military, public assistance participation, and incarceration data. 

An electronic transcript system, the Florida Automated System for Transferring Educational 
Records (FASTER) has been in place since 1988-89. By 1994, Florida had one of the most 
progressive, comprehensive and efficient systems for transferring student records in the 
nation. In 2001, over 900,000 electronic transcripts were exchanged. At about the same 
time that Florida's FASTER system was going into production, Florida began to work with 
other states to develop a nationwide student record transfer system, now known as 
SPEEDE/ExPRESS (Standardization of Postsecondary Education Electronic Data 
Exchange/Exchange of Permanent Records Electronically for Students and Schools). 

The Florida Comprehensive Assessment Test (FCAT) is part of Florida's overall plan to 
increase student achievement by implementing higher standards. The FCAT, administered to 
students in Grades 3-11, contains two basic components: criterion- referenced tests (CRT), 
measuring selected benchmarks in Mathematics, Reading, Science, and Writing from the 
Sunshine State Standards (SSS); and norm- referenced tests (NRT) in Reading and 
Mathematics, measuring individual student performance against national norms. 
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Since 2002, the Florida Education Data Warehouse (EDW) has provided a single repository 
of data extracted from multiple sources available at the state level on students, education 
facilities, curriculum, and instructional staff in the P-20 public education system. The EDW 
allows longitudinal data analysis at the student and staff levels from 1995-96 forward. 
Student level data include demographics, enrollment, course completion, assessment 
results, financial aid, and employment. Future plans include collecting SAT, ACT, and AP 
data, and obtaining information on private school students. 

Future Enhancements 

A new learning and teaching environment tool called Sunshine Connections is under 
development to provide relevant information to educators, administrators, parents and 
students. Teachers will be provided with interactive access to classroom management tools, 
student performance data, and interactive capabilities with other teachers, curricular 
materials, and professional development opportunities. 

Implementation Issues 

o Annual data conferences are held to inform district staff of changes and updates. An 
association of MIS staff meets twice a year to share knowledge and lessons learned, 
and learn of enhancements and useful reports for informing instruction at the local 
level. 

o When new mandates or reporting requirements arise, staff in the FDOE work to 
define, process, and implement the changes in the least painful way for the district, 
so it will be the least burdensome to districts programming. 

o Currently a grant from the Institute of Education Sciences (IES) is being used to 

integrate facilities and finance data in the EDW and to develop a business intelligence 
tool for use with the data warehouse. 


Costs 

Estimated costs to the state: 

o Most data collection systems in Florida have been in existence for decades, thus the 
costs of building them are not relevant to this report. 

o To enhance the data warehouse, the FDOE started with a Request for Information 
(RFI) which resulted in presentations from 15 companies. From this exercise, 
enough information was obtained to create an Invitation to Negotiate (ITN). Since 
the FDOE had no money to build the system, they created a no cost Request for 
Proposal (RFP) - in exchange for providing information about Florida's system, the 
company participating would have "a laboratory to sell to other states." 

o Staff at the FDOE maintain all the systems; six FTEs provide programming support to 
local school districts. 

o In considering costs, it is helpful to view the public education system as a large 

corporation and specifically budget a percentage of the corporation's expenditures for 
information systems rather than look at the specific dollar value associated with each 
component. 

Estimated costs to districts: 

o When FIRN was implemented, districts were not provided software or hardware by 
the state; districts submitted data from their local systems to FIRN in whatever way 
they found to be most expedient. The total costs to districts for that effort are not 
known. However, initial startup funding for this effort was one million dollars 
statewide allocated to districts based on staff FTEs. The development of consortia to 
aid in the new reporting process was part of this effort. 
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o Currently, in every legislative budget a portion of the funding allocated to school 
districts must be used for data and information systems, 
o District representatives from three districts were interviewed to provide anecdotal 
perspective and estimated the staff impact of managing the SSID system in their 
respective districts as: 

o two teams (for student and staff data) of four each (District Size: 30,000 
students, highly mobile) 

o two developers for the staff data and nine developers for the student data, 
ten FTEs to maintain the system (District Size: 128,000 students) 

Benefits and Uses of System 

Benefits 

o Increases in accuracy and efficiencies in data collection have been realized over time, 
o There are efficiencies inherent in state department efforts to define elements such as 
course numbers so that districts do not have to develop these. Standardization 
allows information to transfer from district to district and to higher education with 
shared understanding of how elements are defined, 
o The state has the capability to cross reference data files submitted by the districts 
and identify errors and anomalies, a process not all districts can do locally. Reports 
provided to districts allow errors to be identified and corrected before final 
submission. 

o The FDOE provides data to federal offices (such as the Office for Civil Rights) instead 
of having the districts submit the data directly as is done in most states. 

Uses 

o Data have been used for accountability and reporting in Florida for many years, 
o Data are used for reporting almost immediately after the 2- to 3-week submission 
period. 

o Districts are provided files containing data on their own students who are included in 
calculations for accountability purposes by the state, 
o There is a very high use of data by teachers and administrators, 
o FDOE staff work with legislative staff to ensure or strengthen understanding of the 
data used by legislators. 

o The Florida Office of Program Policy Analysis and Government Accountability uses 
student level data to examine performance in various areas in the context of costs of 
education. 

o The FDOE uses a quid pro quo system for negotiating with researchers requesting 
education data. In exchange for access to the data, the research question either has 
to be one of interest to the state, or fees are assessed for acquiring the data. The 
FDOE requires that agency staff be allowed to review research reports before they 
are released. 

Lessons Learned 

Design 

o Start with an effort sufficiently focused and useful to have "an early win," then build 
on it rather than having a scope that is too broad to be manageable, 
o Do not wait until things are perfect to start. Plan on mid-course corrections and 
phased implementation. 

o Know that all the issues associated with these efforts (matching records across 
systems, confidentiality, demonstrating viability, selling the products, maintaining 
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the quality of operations, watching for pitfalls and opportunities) will need to be 
addressed continually. 

o Identify all the information you want to obtain at the beginning and plan on mid- 
course corrections as dynamics change. Put it in a format you can change because it 
will never remain static. 

o Expect to spend a year in discovery for an effort like a data warehouse and know 
that "no shrink-wrapped answer" is available. Be aware that it will take staff time for 
development and contract monitoring activities. 

o Keep ultimate goals in sight to maintain expectations of where you need to be. 

Staffing 

o Maintain flexibility in the face of changing leadership, technology, expectations, and 
the political environment. 

o As much as possible, maintain continuity of staff. 

o Communicate to staff at the district level that the job entails data quality, not just 
getting the data reported. 

o Hold regular meetings with MIS and program area staff, both within the state 

department and within school systems. Funds for programs are allocated based on 
data, and programs are evaluated under accountability systems. However, it is often 
the MIS staff who are collecting and reporting the data, so communication and 
understanding between these groups is important. 

Maintenance/ Change Control 

o Matching students across collections (for example, enrollment to assessment) and 
within the EDW is done based on a rubric for assigning points of matching multiple 
student-specific variables, such as school of enrollment, first and last names, date of 
birth, and gender. Seventeen rules have been developed to identify a student, with 
varying levels of confidence assigned depending on the combination of matching 
data elements. 

o The FDOE periodically conducts a sunset process to review every data element and 
determine whether it should continue to be in the data collection. Processes are in 
place to track how often elements are used for reporting and analysis. 

o A student locator system using information from the data collections allows districts 
to find all possible matches on a student using a system to match records based on 
similar sounding surnames. 

o The FDOE tries to balance the need to collect all data elements with the goal of 
reducing the reporting burden on districts. 

o When possible, the FDOE reconfigures data at the state level to meet new reporting 
requirements, as that is a less expensive process than asking all districts to re-tool 
their systems. 

o Staff members who are responsible for collecting the data and putting it into the 
warehouse also focus on ensuring the data are clean before it is used by others. 

o Business rules for determining how to handle (for example) changes in a student's 
ethnicity, gender, or program need to be explicated and disseminated. 

o Consult an expert (for example, an economist or statistician) to help decide what to 
do with data anomalies, suspect data that cannot be changed, or data errors that 
affect state totals. 

Partnerships 

o Establish and maintain good relationships with SEA and legislative leadership to 
achieve long-term goals. 
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o Oversight committees need to be reformed as systems change and the scope and 
focus of the project changes. A committee to help design data collection for a P-12 
system will not have the same audience as one focused on data usage, 
o Solicit districts (or universities or community colleges, depending on your effort) 
eager to participate so they can lead the way in telling their colleagues what the 
advantages and payoffs will be. 

o Solicit advocates among your districts. Those willing to go to the legislature can be 
effective. The size of the district is not necessarily a factor in selection. Improving 
data accuracy, resulting in more appropriate funding, can really help a small district, 
o Consider the needs of both small and large districts; neither is immune from the 
pitfalls of data collection and reporting processes, 
o Cooperation and collaboration is enhanced if districts get answers, even unpopular 
ones. 

o Since each system (for example, finance or facilities) has its own business rules, it is 
essential to include all relevant people in the effort to understand how the data are 
being used so as to align processes, definitions and edits. 

Communication 

o Transparency of procedures and processes is helpful, so that all parties involved can 
see the benefits of better quality data, 
o Document all systems and how and why they were designed, 
o In order to improve the quality of data and eliminate confusion, determine one 
source of data for all reports and evaluations, 
o Be aware that once data are linked to other areas, perhaps in ways that were not 
originally intended, this will change the scope of how the data are collected and 
reported and can have unintended consequences. 

Recommendations for Future Development 

o Continue to provide training throughout the state to create a culture of using data to 
make instructional decisions and individualize instruction for all students, 
o Work toward collecting data on why students are not tested, 
o Continue to expand data warehouse capabilities to include facilities and address 
issues of class size. 

o Continue to move toward development of data marts for providing most commonly- 
requested reports from the data warehouse, 
o Work to determine a basic level of information services to districts that can be 
maintained over time regardless of changes in leadership and funding. 
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